Apparatus and method for providing sensory experience

ABSTRACT

Embodiments of the present disclosure relate to a sensory experience providing apparatus for providing a sensory experience based on sound in a vehicle, and a method thereof. The controller is configured to receive a sound played in the vehicle, extract a sound feature from the received sound, generate sensory information based on the extracted sound feature, and provide a sensory experience based on the sensory information.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims the benefit of Korean PatentApplication No. 10-2021-0178963, filed in the Korean IntellectualProperty Office on Dec. 14, 2021, the entire contents of which areincorporated herein by reference.

TECHNICAL FIELD

Embodiments of the present disclosure relate to a sensory experienceproviding apparatus for providing a sensory experience based on sound ina vehicle, and a method thereof

DESCRIPTION OF RELATED ART

Devices with various functions are being applied to vehicles inconsideration of a user's convenience, safety, and interest. One of suchthe devices may be a vibrating seat. The vibrating seat may generatevibrations by controlling a vibration terminal built into a vehicleseat. A conventional vibrating seat may generate vibrations inconjunction with the sound of a sound source being played or maygenerate vibrations corresponding to dangerous situations when variousdangerous situations occur.

Conventionally, a vibrator excitation method applied to a seat and asoft actuator excitation method applied to a steering wheel have beendisclosed.

SUMMARY

Embodiments of the present disclosure have been made to solve theabove-mentioned problems occurring in the existing technology whileadvantages achieved by the existing technology are maintained intact.

An embodiment of the present disclosure provides a sensory experienceproviding apparatus for providing sensory information such as olfactoryinformation and/or visual information based on a sound generated by avehicle, and a method thereof.

The technical problems to be solved by embodiments of the presentdisclosure are not limited to the aforementioned problems, and any othertechnical problems not mentioned herein will be clearly understood fromthe following description by those skilled in the art to whichembodiments of the present disclosure pertain.

According to an embodiment of the present disclosure, a method forproviding a sensory experience includes receiving, by a controller, asound played in a vehicle, extracting, by the controller, a soundfeature from the received sound, generating, by the controller, sensoryinformation based on the extracted sound feature, and providing, by thecontroller, a sensory experience based on the sensory information.

The receiving of the sound includes receiving, by the controller, atleast one of an acceleration sound or a cornering sound.

The extracting of the sound feature includes extracting, by thecontroller, at least one of sound amplitude, a sound frequency, or asound tone from the received sound.

The generating of the sensory information includes classifying, by thecontroller, a sound type by analyzing the extracted sound feature andgenerating, by the controller, at least one of olfactory information orvisual information, which matches the classified sound type.

The olfactory information includes at least one of a fragrance type, afragrance concentration, a fragrance diffusion location, a fragrancediffusion pattern, or a fragrance diffusion time.

The olfactory information further includes at least one of an oxygensupply location, an oxygen supply amount, or an oxygen supply pattern.

The visual information includes at least one of a visual informationtype, a lighting color, lighting brightness, alighting pattern, an imagetype, or a color change order according to a vehicle speed.

The providing of the sensory experience includes adjusting, by thecontroller, an oxygen supply of an oxygen generator based on the sensoryinformation and adjusting, by the controller, fragrance diffusion of afragrance generator based on the sensory information.

The providing of the sensory experience includes controlling, by thecontroller, an operation of a lighting based on the sensory informationand outputting, by the controller, an output of a display based on thesensory information.

According to an embodiment of the present disclosure, a sensoryexperience providing apparatus includes a controller mounted in avehicle. The controller is configured to receive a sound played in thevehicle, extracts a sound feature from the received sound, generatessensory information based on the extracted sound feature, and provides asensory experience based on the sensory information.

The sound is at least one of an acceleration sound or a cornering sound.

The sound feature includes at least one of sound amplitude, a soundfrequency, or a sound tone.

The controller is configured to classify a sound type by analyzing theextracted sound feature and generates at least one of olfactoryinformation or visual information, which matches the classified soundtype.

The olfactory information includes at least one of a fragrance type, afragrance concentration, a fragrance diffusion location, a fragrancediffusion pattern, or a fragrance diffusion time.

The olfactory information includes at least one of an oxygen supplylocation, an oxygen supply amount, or an oxygen supply pattern.

The visual information includes at least one of a visual informationtype, a lighting color, lighting brightness, alighting pattern, an imagetype, and a color change order according to a vehicle speed.

The sensory experience providing apparatus further includes an oxygengenerator configured to supply oxygen or a fragrance generatorconfigured to supply a fragrance.

The controller is configured to adjust an oxygen supply of the oxygengenerator based on the sensory information and adjusts fragrancediffusion of the fragrance generator based on the sensory information.

The sensory experience providing apparatus further includes lighting anda display. The controller is configured to control an operation of thelighting based on the sensory information and controls an output of thedisplay based on the sensory information.

The controller is configured to separate the received sound into a musicsound and a virtual sound, determines a sound source style of the musicsound, compares and analyzes a volume of the music sound with a volumeof the virtual sound, and generates the sensory information based on thesound source style and the volume comparison result.

As discussed, the method and apparatus suitably include use of acontroller or processer.

In another aspect, vehicles are provided that comprise an apparatus asdisclosed herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of embodiments ofthe present disclosure will be more apparent from the following detaileddescription taken in conjunction with the accompanying drawings:

FIG. 1 is a block diagram illustrating a sensory experience providingapparatus, according to an exemplary embodiment of the presentdisclosure;

FIG. 2 is a diagram for describing a process of setting an emotionalcare function, according to embodiments of the present disclosure;

FIG. 3 is a view for describing a sensory substitution process,according to embodiments of the present disclosure;

FIG. 4 is a flowchart illustrating a sensory experience providingmethod, according to a first embodiment of the present disclosure;

FIG. 5 is a flowchart illustrating a sensory experience providingmethod, according to a second embodiment of the present disclosure;

FIG. 6 is a flowchart illustrating a sensory experience providingmethod, according to a third embodiment of the present disclosure; and

FIG. 7 is a block diagram illustrating a computing system performing asensory experience providing method, according to an exemplaryembodiment of the present disclosure.

DETAILED DESCRIPTION

It is understood that the term “vehicle” or “vehicular” or other similarterm as used herein is inclusive of motor vehicles in general such aspassenger automobiles including sports utility vehicles (SUV), buses,trucks, various commercial vehicles, and the like, and includes hybridvehicles, electric vehicles, plug-in hybrid electric vehicles. Asreferred to herein, a hybrid vehicle is a vehicle that has two or moresources of power, for example both gasoline-powered and electric-poweredvehicles.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the disclosure.As used herein, the singular forms “a,” “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof. As used herein, the term“and/or”includes any and all combinations of one or more of theassociated listed items. Throughout the specification, unless explicitlydescribed to the contrary, the word “comprise” and variations such as“comprises” or “comprising” will be understood to imply the inclusion ofstated elements but not the exclusion of any other elements. Inaddition, the terms “unit”, “-er”, “-of”, and “module” described in thespecification mean units for processing at least one function andoperation and can be implemented by hardware components or softwarecomponents and combinations thereof.

Further, the control logic of embodiments of the present disclosure maybe embodied as non-transitory computer readable media on a computerreadable medium containing executable program instructions executed by aprocessor, controller, or the like. Examples of computer readable mediainclude, but are not limited to, ROM, RAM, compact disc (CD)-ROMs,magnetic tapes, floppy disks, flash drives, smart cards, and opticaldata storage devices. The computer readable medium can also bedistributed in network coupled computer systems so that the computerreadable media is stored and executed in a distributed fashion, e.g., bya telematics server or a Controller Area Network (CAN).

Hereinafter, some embodiments of the present disclosure will bedescribed in detail with reference to the accompanying drawings. Inadding reference numerals to components of each drawing, it should benoted that the same components have the same reference numerals,although they are indicated on another drawing. Furthermore, indescribing the embodiments of the present disclosure, detaileddescriptions associated with well-known functions or configurations willbe omitted when they may make subject matters of embodiments of thepresent disclosure unnecessarily obscure.

In describing elements of exemplary embodiments of the presentdisclosure, the terms first, second, A, B, (a), (b), and the like may beused herein. These terms are only used to distinguish one element fromanother element, but do not limit the corresponding elementsirrespective of the nature, order, or priority of the correspondingelements. Furthermore, unless otherwise defined, all terms includingtechnical and scientific terms used herein are to be interpreted as iscustomary in the art to which embodiments of the present disclosurebelong. It will be understood that terms used herein should beinterpreted as having a meaning that is consistent with their meaning inthe context of embodiments of the present disclosure and the relevantart and will not be interpreted in an idealized or overly formal senseunless expressly so defined herein.

In this specification, a technology is presented to provide a sensoryexperience (e.g., mood lighting and/or fragrance) based on a sound(e.g., emotional sound and/or virtual sound) generated (played) in avehicle by using a sensory substitution technology.

FIG. 1 is a block diagram illustrating a sensory experience providingapparatus, according to an exemplary embodiment of the presentdisclosure.

Referring to FIG. 1 , a sensory experience providing apparatus 100 mayinclude a sound detector 110, an olfactory transmitter 120, a visualtransmitter 130, and a controller 140.

The sound detector 110 may detect a sound generated in a vehicle. Thesound detector 110 may detect an emotional sound played by a mediaplayback device (not shown) and/or a virtual sound (e.g., anacceleration sound, a cornering sound, a driving sound, and the like)played by a sound generating device (not shown). The sound detector 110may detect a sound output through a speaker (not shown) by using a soundsensor (e.g., a microphone, or the like).

The olfactory transmitter 120 may be a device for providing an olfactoryexperience and may include an oxygen generator 121 and a fragrancegenerator 122. The oxygen generator 121 may be mounted in an engine roomand/or a trunk of the vehicle. The oxygen generator 121 may be connectedto an air conditioning system by a duct. The oxygen generator 121 maysupply oxygen to the interior of the vehicle through the airconditioning system. The oxygen generator 121 may adjust an oxygensupply location, an oxygen supply amount, and oxygen supply timing underthe control of the controller 140. The fragrance generator 122 may bepositioned on a console or the like and may be portable. The fragrancegenerator 122 may be implemented to have a capsule filling type or aspray type. The fragrance generator 122 may adjust a fragrance type, afragrance concentration, a fragrance diffusion location, a fragrancediffusion pattern, a fragrance diffusion time depending on controlcommands of the controller 140.

The visual transmitter 130 may be a device for providing a visualexperience and may include a lighting 131 and a display 132. Thelighting 131 may be installed as mood lighting at a specific locationpredetermined in a vehicle or may be installed along a border line of acrush pad. The lighting 131 may be implemented with a light emittingdiode (LED) or the like. The lighting 131 may control a lighting color,lighting brightness, and a lighting pattern (turning on or offlighting). The display 132 may display an image. The display 132 mayinclude at least one of a head-up display (HUD), a liquid crystaldisplay (LCD), a thin film transistor-liquid crystal display (TFT-LCD),an organic light-emitting diode (OLED) display, a flexible display, athree-dimensional (3D) display, a transparent display, a touch screen,and a cluster.

The controller 140 may be configured to be electrically connected to thesound detector 110, the olfactory transmitter 120, and the visualtransmitter 130. The controller 140 may be configured to include aprocessor 141 and a memory 142. The processor 141 may control overalloperations of the controller 140. The processor 141 may be implementedwith at least one of processing devices such as an application specificintegrated circuit (ASIC), a digital signal processor (DSP), aprogrammable logic device (PLD), a field programmable gate array (FPGA),a central processing unit (CPU), a microcontroller and/or amicroprocessor. The memory 142 may store a sensory substitutionalgorithm, a feature extraction algorithm, and various settinginformation. The memory 142 may be a non-transitory storage medium thatstores instructions executed by the processor 141. The memory 142 may beimplemented with at least one of storage media such as a flash memory, ahard disk, a solid state disk (SSD), a secure digital (SD) card, arandom access memory (RAM), a static random access memory (SRAM), a readonly memory (ROM), a programmable read only memory (PROM), anelectrically erasable and programmable ROM (EEPROM), an erasable andprogrammable ROM (EPROM) and/or web storage. The memory 142 may bepositioned inside and/or outside the processor 141.

The processor 141 may perform signal conversion for converting thecorresponding sound into an electric signal when a sound, for example,an emotional sound and/or a virtual sound is played in the vehicle. Theprocessor 141 may substitute (convert) the sound played in the vehicleinto sensory information by using the sensory substitution algorithm.The sensory substitution algorithm may include steps of sensation,perception, and recognition. A sensation step is a process of convertinga physical stimulus (e.g., sound) into an electrical signal; aperception step is a process of extracting a meaningful pattern from theelectrical signal converted in the sensation step and giving meaning tothe meaningful pattern; and a recognition step is a process ofexpressing emotional meaning by using the five senses. The sensoryinformation may include olfactory information and/or visual information.The olfactory information may include a fragrance type (e.g., floral andaromatic), a fragrance concentration (strength), a fragrance diffusionlocation, a fragrance diffusion pattern, and/or a fragrance diffusiontime. In addition, the olfactory information may include an oxygensupply amount, an oxygen supply location, and an oxygen supply pattern.The visual information may include a visual information type (e.g.,display color, mood lighting, and/or images), a color, illuminance(brightness), and/or an image type (e.g., an aurora image and/or aforest image).

The processor 141 may convert sensory information into a sensory signal(e.g., an olfactory signal and/or a visual signal). In other words, theprocessor 141 may control the olfactory transmitter 120 and/or thevisual transmitter 130 based on the sensory information and then mayoutput an olfactory signal (e.g., oxygen, fragrance, and the like)and/or a visual signal (e.g., mood lighting, HUD, and/or crush padlighting).

For example, when a driver is sleeping soundly or a sound source havinga depressing atmosphere is played while the vehicle is stopped, theprocessor 141 may control the fragrance generator 122 so as to diffusethe scent of gardenia having which has a calming effect on nerves.

As another example, when a sound source having a strong beat is playedin driving situations, the processor 141 may allow the fragrancegenerator 122 to diffuse the scent of lavender that helps relievestress.

As another example, when an acceleration sound or a cornering sound isplayed in driving situations, the processor 141 may control thefragrance generator 122 to diffuse the scent of lavender depending onpredetermined information.

The processor 141 may recognize a sound source by determining thestopped or driving state of the vehicle. Furthermore, when a drivingsound is played, the processor 141 may recognize the driving state inconnection with a vehicle speed and/or an opening degree of anaccelerator pedal.

The processor 141 may generate the visual information based on a vehiclestatus and sound. The visual information may include a visualinformation type, turning on/off mood lighting, a lighting color,lighting brightness, an image type, and/or a color change orderaccording to a vehicle speed.

For example, the processor 141 may control the display 132 so as toadjust a screen color depending on color information matching a sound.For example, when music corresponds to a quiet sound source, theprocessor 141 may provide the display 132 with yellow and purple colorsas color information; when the music corresponds to a relaxation soundsource, the processor 141 may provide the display 132 with green,indigo, and blue colors as color information; and, when the musiccorresponds to a healing sound source, the processor 141 may provide thedisplay 132 with orange and red colors as color information. When adriving virtual sound is played, the processor 141 may control thedisplay 132 so as to change a screen color to a color matching a vehiclespeed.

As another example, the processor 141 may play an image matching soundand then may output the image to the display 132. For example, theprocessor 141 may play various aurora images depending on a sound andmay output the various aurora images to the display 132.

As another example, the processor 141 may control the lighting 131depending on a sound and driving type. For example, the processor 141may change colors of crush pad lighting, center console lighting, anddoor trim lighting depending on a sound bit and/or driving type. At thistime, the crush pad lighting, the center console lighting, and the doortrim lighting may be adjusted with different colors and brightness.

The processor 141 may determine emotion by analyzing emotion valence,emotion arousal, and emotion sensitivity based on the sound. Theprocessor 141 may classify a sound into three types based on moodvectoring (an emotion analysis model). The sound type may be dividedinto a quiet sound source, a stable sound source, and a healing soundsource. In other words, when the sound-based emotion analysis result ischanged from depression or anger to happiness, the processor 141 mayclassify the sound type as a quiet sound source; when the sound-basedemotion analysis result is changed from melancholy to calm, theprocessor 141 may classify the sound type as a stable sound source; and,when the sound-based emotion analysis result is changed from anger tocalm, the processor 141 may classify the sound type as a healing soundsource.

The processor 141 may determine a driving pattern by analyzing theopening degree of an accelerator pedal when providing a sound-basedolfactory experience. The processor 141 may determine fragrancediffusion timing based on the determined driving pattern.

Moreover, when providing a sound-based visual experience, the processor141 may determine the driving environment, for example, daytime,nighttime, tunnel, clear weather, and traffic flow, through the analysisof the illumination around the vehicle. The processor 141 may provide avisual experience by adjusting visual substitution according to thedetermined driving environment.

FIG. 2 is a diagram for describing a process of setting an emotionalcare function, according to embodiments of the present disclosure.

The controller 140 may be configured to output an environment settingscreen 210 to the display 132 depending on a user input. When anemotional care item is selected in the environment setting screen 210,the controller 140 may be configured to display an emotional caresetting screen 220 on the display 132. When a fragrance item is selectedfrom the emotional care setting screen 220, the controller 140 may beconfigured to output the fragrance setting screen 230 to the display132. The fragrance setting screen 230 may include a location at which afragrance is capable of being supplied, a location at which oxygen iscapable of being supplied, and a fragrance supply strength adjustmentbar. The controller 140 may be configured to set fragrance supplystrength to strong, medium, or weak strength depending on themanipulation of the fragrance supply strength adjustment bar. Thecontroller 140 may be configured to set a location, which is selected bya user, from among locations at which a fragrance is capable of beingsupplied, as a fragrance diffusion location. Moreover, the controller140 may be configured to set a location, which is selected by the user,from among locations at which oxygen is capable of being supplied, as anoxygen supply location.

FIG. 3 is a view for describing a sensory substitution process,according to embodiments of the present disclosure.

The controller 140 may be configured to receive a sound generated by avehicle in S110. Through the sound detector 110, the controller 140 maybe configured to receive (detect) a sound played by a media playbackdevice (not shown) or a sound generated by a sound generating device(not shown). The controller 140 may be also configured to receivefeedback on a sound output from a speaker by using a microphone or thelike.

The controller 140 may be configured to extract a sound feature from thereceived sound in S120. The sound feature may include sound strength(amplitude), sound pitch (frequency), and/or sound tone (sound wavepattern). The controller 140 may be configured to classify (determine) asound type based on the extracted sound feature. For example, thecontroller 140 may be configured to determine whether the played soundis a quiet sound source, a stable sound source, or a healing soundsource, by analyzing the extracted sound feature.

The controller 140 may be configured to convert the received sound intosensory information based on the extracted sound feature in S130. Inother words, the controller 140 may be configured to generate thesensory information matching the classified sound type. For example,when the received sound is a quiet sound source, a stable sound source,and/or a healing sound source, the controller 140 may be configured tomatch a stable floral and/or aromatic fragrance with the received sound.Besides, when the received sound is a quiet sound source, a stable soundsource, and/or a healing sound source, the controller 140 may beconfigured to select a lighting color, lighting brightness(illuminance), and/or display image, which corresponds to the receivedsound.

The controller 140 may be configured to deliver a sense based on theconverted sensory information in S140. For example, the controller 140may be configured to control the fragrance generator 122 so as to adjustthe scent of gardenia capable of giving a sense of stability and/or thescent of lavender capable of relieving stress. As another example, thecontroller 140 may be configured to control the lighting 131 so as toadjust a mood lighting color.

FIG. 4 is a flowchart illustrating a sensory experience providingmethod, according to a first embodiment of the present disclosure.

The controller 140 may be configured to receive an emotional soundplayed in the vehicle in S210. The sound detector 110 may directlyreceive an emotional sound played by a media playback device (not shown)or may detect an emotional sound, which is output to the outside througha speaker, by using a microphone or the like. The sound detector 110 maytransmit the detected emotional sound to the controller 140. Thecontroller 140 may be configured to receive an emotional soundtransmitted from the sound detector 110.

The controller 140 may be configured to extract a sound feature from theemotional sound in S220. The controller 140 may be configured to extractthe sound feature by using a feature extraction algorithm stored in thememory 142. The sound feature may include the amplitude, frequency,and/or tone of the sound.

The controller 140 may be configured to generate sensory informationbased on the extracted sound feature in S230. The controller 140 may beconfigured to determine the emotional state by analyzing the extractedsound feature. The controller 140 may be configured to classify theplayed emotional sound depending on the emotional state. That is, thecontroller 140 may be configured to classify (determine) a sound type(sound source type) of the emotional sound. The controller 140 may beconfigured to generate the sensory information matching the classifiedsound type. For example, when the played emotional sound is classifiedas a healing sound source, the controller 140 may be configured togenerate olfactory information and visual information, which match thehealing sound source. The olfactory information may include a fragrancetype, a fragrance concentration, a fragrance diffusion location, afragrance diffusion pattern, and/or a fragrance diffusion time. Theolfactory information may include an oxygen supply location, an oxygensupply amount, and/or an oxygen supply pattern. The visual informationmay include a visual information type, alighting color, lightingbrightness, alighting pattern, alighting location, and/or an image type.

The controller 140 may be configured to provide a sensory experiencebased on sensory information in S240. The controller 140 may beconfigured to control the operation of the olfactory transmitter 120and/or the visual transmitter 130 based on the sensory information. Thecontroller 140 may be configured to adjust an oxygen supply through theoxygen generator 121 and may control the fragrance generator 122 so asto adjust fragrance diffusion. The controller 140 may be configured toadjust the color, brightness, and/or flickering pattern of the lighting131, and may control the screen color and/or displayed image of thedisplay 132.

FIG. 5 is a flowchart illustrating a sensory experience providingmethod, according to a second embodiment of the present disclosure.

The controller 140 may be configured to receive a virtual soundgenerated by a vehicle in S310. The controller 140 may be configured toreceive the virtual sound detected by the sound detector 110. The sounddetector 110 may detect at least one of virtual sounds such as anacceleration sound, a cornering sound, and/or a driving sound played bya sound generating device. The sound detector 110 may transmit thedetected virtual sound to the controller 140. The controller 140 may beconfigured to receive a virtual sound transmitted by the sound detector110.

The controller 140 may be configured to extract a sound feature from thevirtual sound in S320. The sound feature may include the amplitude,frequency, and/or tone of the sound.

The controller 140 may be configured to generate sensory informationbased on the sound feature and driving information in S330. Thecontroller 140 may be configured to determine a sound type of thevirtual sound by analyzing the extracted sound feature. Furthermore, thecontroller 140 may be configured to obtain driving information by usingother electronic control units (ECUs) and/or sensors in the vehicle. Thedriving information may include a vehicle status (e.g., stop, driving,and the like), a vehicle speed, and a driving type (e.g., constantspeed, acceleration, and the like). The controller 140 may be configuredto generate sensory information (i.e., information for controllingoperations of the olfactory transmitter 120 and/or the visualtransmitter 130) for implementing a sensory experience based on a soundfeature and/or driving information.

The controller 140 may be configured to provide a sensory experiencebased on sensory information in S340. The controller 140 may beconfigured to control the operation of the olfactory transmitter 120and/or the visual transmitter 130 based on the sensory information. Theolfactory transmitter 120 may adjust the supply of oxygen and/orfragrance under the control of the controller 140. On the basis of theinstruction of the controller 140, the visual transmitter 130 maycontrol an operation of the lighting 131 and may control a screen colorof the display 132 and an image displayed on the display 132.

FIG. 6 is a flowchart illustrating a sensory experience providingmethod, according to a third embodiment of the present disclosure.

The controller 140 may be configured to receive a sound played in avehicle. The controller 140 may be configured to separate the receivedsound into a music sound and a virtual sound.

The controller 140 may be configured to determine a sound source style(a sound type) of the separated music sound in S410. The sound sourcestyle may be classified into a sound source that soothes depressed mood,a sound source that relieves anger control disorders, a sound sourcethat provides a pleasant healing feeling.

The controller 140 may be configured to compare and analyze the volumeof the separated music sound with the volume of the virtual sound inS420. The controller 140 may be configured to calculate a sound-to-musicratio (SMR) of the music sound and the virtual sound by using Equation1.

$\begin{matrix}{{SMR} = {20{\log_{10}\left( \frac{A_{sound}}{A_{music}} \right)}}} & \left\lbrack {{Equation}1} \right\rbrack\end{matrix}$

Here, A_(sound) denotes the volume of a virtual sound; and A_(music)denotes the volume of a music sound.

The controller 140 may be configured to determine a vehicle status asstop, low speed, constant speed, or acceleration based on the calculatedSMR For example, when the SMR is less than −3 dB, the controller 140 maybe configured to determine that the vehicle is highly likely to be in astop state and/or low-speed state. When the SMR is 0±3 dB, thecontroller 140 may be configured to determine that the vehicle is highlylikely to be in a constant speed state. When the SMR exceeds 3 dB, thecontroller 140 may be configured to determine that the vehicle is highlylikely to be in an accelerated state.

The controller 140 may be configured to generate sensory substitutioninformation for selecting a type of a fragrance or lighting based on themusic sound information and virtual sound information in S430. Thecontroller 140 may be configured to generate excitation patterninformation of the scent or lighting based on the determined music styleand the volume comparison result.

The controller 140 may be configured to provide a sensory experienceutilizing five senses based on the sensory substitution technology inS440. The controller 140 may be configured to control the operation ofthe olfactory transmitter 120 and/or the visual transmitter 130 based onthe generated excitation pattern information. The olfactory transmitter120 may adjust the supply of oxygen and/or fragrance under the controlof the controller 140. On the basis of the instruction of the controller140, the visual transmitter 130 may control an operation of the lighting131 and may control a screen color of the display 132 and an imagedisplayed on the display 132.

FIG. 7 is a block diagram illustrating a computing system performing asensory experience providing method, according to an exemplaryembodiment of the present disclosure.

Referring to FIG. 7 , a computing system 1000 may include at least oneprocessor 1100, a memory 1300, a user interface input device 1400, auser interface output device 1500, a storage 1600, and a networkinterface 1700, which are connected with each other via a bus 1200.

The processor 1100 may be a central processing unit (CPU) or asemiconductor device that processes instructions stored in the memory1300 and/or the storage 1600. Each of the memory 1300 and the storage1600 may include various types of volatile or nonvolatile storage media.For example, the memory 1300 may include a read only memory (ROM) 1310and a random access memory (RAM) 1320.

Accordingly, the operations of the method or algorithm described inconnection with the embodiments disclosed in the specification may bedirectly implemented with a hardware module, a software module, or acombination of the hardware module and the software module, which isexecuted by the processor 1100. The software module may reside on astorage medium (i.e., the memory 1300 and/or the storage 1600) such as arandom access memory (RAM), a flash memory, a read only memory (ROM), anerasable and programmable ROM (EPROM), an electrically EPROM (EEPROM), aregister, a hard disk drive, a removable disc, or a compact disc-ROM(CD-ROM). The exemplary storage medium may be coupled to the processor1100. The processor 1100 may read out information from the storagemedium and may write information in the storage medium. Alternatively,the storage medium may be integrated with the processor 1100. Theprocessor 1100 and storage medium may be implemented with an applicationspecific integrated circuit (ASIC). The ASIC may be provided in a userterminal. Alternatively, the processor 1100 and storage medium may beimplemented with separate components in the user terminal.

The above description is merely an example of the technical idea ofembodiments of the present disclosure, and various modifications andmodifications may be made by one skilled in the art without departingfrom the essential characteristic of embodiments of the presentdisclosure. Therefore, embodiments of the present disclosure are notintended to limit the technical spirit of embodiments of the presentdisclosure but provided only for the illustrative purpose. The scope ofprotection of embodiments of the present disclosure should be construedby the attached claims, and all equivalents thereof should be construedas being included within the scope of embodiments of the presentdisclosure.

According to an exemplary embodiment of the present disclosure, sensoryinformation such as olfactory information and/or visual information isgenerated based on the sound generated by a vehicle, thereby providing auser with emotional care using a sensory experience.

Hereinabove, although embodiments of the present disclosure have beendescribed with reference to exemplary embodiments and the accompanyingdrawings, embodiments of the present disclosure are not limited thereto,but may be variously modified and altered by those skilled in the art towhich embodiments of the present disclosure pertain without departingfrom the spirit and scope of embodiments of the present disclosureclaimed in the following claims.

What is claimed is:
 1. A method for providing a sensory experience, themethod comprising: receiving, by a controller, a sound played in avehicle; extracting, by the controller, a sound feature from thereceived sound; generating, by the controller, sensory information basedon the extracted sound feature; and providing, by the controller, asensory experience based on the sensory information.
 2. The method ofclaim 1, wherein the receiving of the sound comprises: receiving, by thecontroller, at least one of an acceleration sound or a cornering sound.3. The method of claim 1, wherein the extracting of the sound featurecomprises: extracting, by the controller, at least one of a soundamplitude, a sound frequency, or a sound tone from the received sound.4. The method of claim 1, wherein the generating of the sensoryinformation comprises: classifying, by the controller, a sound type byanalyzing the extracted sound feature; and generating, by thecontroller, at least one of olfactory information or visual information,which matches the classified sound type.
 5. The method of claim 4,wherein the olfactory information comprises at least one of a fragrancetype, a fragrance concentration, a fragrance diffusion location, afragrance diffusion pattern, or a fragrance diffusion time.
 6. Themethod of claim 5, wherein the olfactory information further comprisesat least one of an oxygen supply location, an oxygen supply amount, oran oxygen supply pattern.
 7. The method of claim 4, wherein the visualinformation comprises at least one of a visual information type, alighting color, lighting brightness, a lighting pattern, an image type,or a color change order according to a vehicle speed.
 8. The method ofclaim 1, wherein the providing of the sensory experience comprises:adjusting, by the controller, an oxygen supply of an oxygen generatorbased on the sensory information; and adjusting, by the controller,fragrance diffusion of a fragrance generator based on the sensoryinformation.
 9. The method of claim 1, wherein the providing of thesensory experience comprises: controlling, by the controller, anoperation of alighting based on the sensory information; and outputting,by the controller, an output of a display based on the sensoryinformation.
 10. A sensory experience providing apparatus, the apparatuscomprising a controller mounted in a vehicle, wherein the controller isconfigured to: receive a sound played in the vehicle; extract a soundfeature from the received sound; generate sensory information based onthe extracted sound feature; and provide a sensory experience based onthe sensory information.
 11. The apparatus of claim 10, wherein thesound is at least one of an acceleration sound or a cornering sound. 12.The apparatus of claim 10, wherein the sound feature comprises at leastone of a sound amplitude, a sound frequency, or a sound tone.
 13. Theapparatus of claim 10, wherein the controller is further configured to:classify a sound type by analyzing the extracted sound feature; andgenerate at least one of olfactory information or visual information,which matches the classified sound type.
 14. The apparatus of claim 13,wherein the olfactory information comprises at least one of a fragrancetype, a fragrance concentration, a fragrance diffusion location, afragrance diffusion pattern, or a fragrance diffusion time.
 15. Theapparatus of claim 14, wherein the olfactory information comprises atleast one of an oxygen supply location, an oxygen supply amount, or anoxygen supply pattern.
 16. The apparatus of claim 13, wherein the visualinformation comprises at least one of a visual information type, alighting color, lighting brightness, a lighting pattern, an image type,or a color change order according to a vehicle speed.
 17. The apparatusof claim 10, further comprising: an oxygen generator configured tosupply oxygen; and a fragrance generator configured to supply afragrance.
 18. The apparatus of claim 17, wherein the controller isfurther configured to: adjust an oxygen supply of the oxygen generatorbased on the sensory information; and adjust fragrance diffusion of thefragrance generator based on the sensory information.
 19. The apparatusof claim 10, further comprising: alighting; and a display, wherein thecontroller is further configured to: control an operation of thelighting based on the sensory information; and control an output of thedisplay based on the sensory information.
 20. The apparatus of claim 19,wherein the controller is further configured to: separate the receivedsound into a music sound and a virtual sound; determine a sound sourcestyle of the music sound; compare and analyze a volume of the musicsound with a volume of the virtual sound; and generate the sensoryinformation based on the sound source style and the volume comparisonresult.